1. Field of the Invention
Embodiments of the present invention relate to computer-implemented methods and systems for generating, maintaining and managing software testing documentation and the results generated by such testing.
2. Description of the Prior Art and Related Information
As software becomes increasingly complex, the tools and framework for testing the software must keep pace. The functionality of the software must be thoroughly tested to ensure that it performs as designed, to ensure that it is stable and that it smoothly interoperates with other software applications and modules in its intended working environment. This testing is the job of Quality Assurance.
As Product Marketing defines release features and Engineering implements them, Quality Assurance (QA) writes test cases to validate these features. Conventionally, QA engineers develop test plans that test predetermined portions of the functionality of a piece of software. Conventionally, such test plans are static files such as, for example, Microsoft Excel® or Microsoft Word®. As the QA engineer tests the software according to the developed test plans, he or she may manually enter the test results of such testing into these static files. For example, the QA engineer may enter whether the particular radio buttons behaved as expected, whether expected values were indeed obtained, or whether each step passes or fails a predetermined test. As the testing process continues and several iterations of the testing process are carried out, a great many such static files may be generated. Such static files containing the test results can be difficult to manage, especially as the number of such files grows large. It is also difficult to track changes over time using such static files, as the number of tests changes with each release of the software under test. Moreover, when these static files are generated, it is difficult to precisely identify the scope of each test cycle, as each test cycle requires test cases to be updated with the appropriate information such as, for example, the status of test runs, bug numbers and blocking issues. Lastly, such conventional individual static files make it increasingly difficult to collect statistics and to monitor the progress of the most basic QA activities such as, for example, QA test planning and QA testing.
There have been some efforts by software companies to solve the aforementioned test documentation management and test cycle scope management. Such efforts have included managing all documentation in such static or flat files, and using test management systems, such as the package “TestDirector” by Mercury Interactive of Mountain View, Calif. However, this approach (and others like it) is not believed to enable to define and freeze the test scope for every test cycle—which is paramount if meaningful comparisons are to be made across testing cycles. None of these solutions are believed to enable the collection of statistics and the generation of statistics for an isolated test cycle.